Skip to main content

Advertisement

SpringerLink
Log in

A study of gender in user reviews on the Google Play Store

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

User reviews that are posted on the Google Play Store provide app developers with important information such as bug reports, feature requests, and user experience. Developers should maintain their apps while taking user feedback into account to succeed in the competitive market of mobile apps. the Google Play Store provides a star-rating mechanism for users to rate apps on a scale of one to five. Apps that are ranked higher and have higher star ratings are more likely to be downloaded. In this paper, we investigate and compare men’s and women’s participation in user reviews that are posted on the Google Play Store. We analyze 438,707 user reviews of the top 156 Android apps over six months. We find that women give higher star ratings and use more positive sentiment in their reviews than men. Furthermore, women’s reviews receive more likes and are ranked higher in the top 10 by the Google Play Store. For the reviews from which user gender can be inferred, we find that men submit more reviews than women, making reviews by men more likely to be visible to app developers and other users. Past research has shown that app developers respond more to negative reviews with fewer stars. We found that developers respond to a greater percentage of men’s reviews than women’s. The small number of and more positive reviews by women are less likely to be addressed by app developers; thus, the resulting changes to apps will align more with the needs of men users, perhaps causing even less participation by women in the Google Play Store reviews. Our findings suggest that developers should take gender into consideration when responding to reviews to help mitigate a feedback loop of bias. Our observations also suggest a need for future research in this area to understand the motivations of men and women in reviewing apps and how developers respond to reviews.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://play.google.com/

  2. http://www.stackoverflow.com/

  3. http://www.github.com/

  4. http://seleniumhq.org/

  5. http://www.twitter.com/

  6. http://www.dribbble.com/

  7. https://play.google.com/store/apps/details?id=org.twisevictory.apps

  8. https://play.google.com/store/apps/details?id=com.mvas.stb.emu.pro

  9. http://www.facebook.com/

  10. http://www.drupal.org/

  11. http://www.wordpress.com/

  12. https://www.taskrabbit.com/

  13. https://www.fiverr.com/

References

  • Argamon S, Koppel M, Fine J, Shimoni A R (2003) Gender, genre, and writing style in formal written texts. Text-The Hague Then Amsterdam Then Berlin 23(3):321–346

    Google Scholar 

  • Asiri O, Chang CK (2018) Investigating users’ experiences and attitudes towards mobile apps’ reviews. In: International Conference on Human-Computer Interaction. Springer, pp 481–499

  • Baek Y (2013) Analysis of user’s attitude toward apps, intention to use and continual consuming intention-focused on mobile commerce. Int J Content 9(4):35–44

    Article  Google Scholar 

  • Bakar N S A A, Mahmud I (2013) Empirical analysis of android apps permissions. In: Proceedings of the 2013 International Conference on Advanced Computer Science Applications and Technologies (ACSAT). IEEE, pp 406–411

  • Basili V R (1992) Software modeling and measurement: the goal/question/metric paradigm. Technical report, Institute for advanced computer studies

  • Bavota G, Linares-Vasquez M, Bernal-Cardenas C E, Penta M D, Oliveto R, Poshyvanyk D (2015) The impact of api change-and fault-proneness on the user ratings of android apps. IEEE Trans Softw Eng 41(4):384–407

    Article  Google Scholar 

  • Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann stat:1165–1188

  • Blei D M, Ng A Y, Jordan M I (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  • Bolboacȧ S D, Jäntschi L, Sestraṡ A F, Sestraṡ R E, Pamfil D C (2011) Pearson-fisher chi-square statistic revisited. Information 2(3):528–545

    Article  Google Scholar 

  • Bonnington C (2013) Are men and women using mobile apps differently? do men and women have appreciably different tastes in apps? [Online]. Available: https://www.wired.com/2013/04/men-women-app-usage/

  • Burger JD, Henderson J, Kim G, Zarrella G (2011) Discriminating gender on twitter. In: Proceedings of the conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 1301–1309

  • Collier B, Bear J (2012) Conflict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, pp 383–392

  • De Marneffe M C, MacCartney B, Manning C D, et al. (2006) Generating typed dependency parses from phrase structure parses. In: 5Th international conference on language resources and evaluation, vol 6, pp 449–454

  • Deveaud R, SanJuan E, Bellot P (2014) Accurate and effective latent concept modeling for ad hoc information retrieval. Doc Numér 17(1):61–84

    Article  Google Scholar 

  • Fink C, Kopecky J, Morawski M (2012) Inferring gender from the content of tweets: a region specific example. In: Sixth International AAAI Conference on Weblogs and Social Media

  • Foltz PW, Laham D, Landauer TK (1999) Automated essay scoring: Applications to educational technology. In: EdMedia+ innovate learning, Association for the Advancement of Computing in Education (AACE), pp 939–944

  • Ford D, Harkins A, Parnin C (2017) Someone Like me: How does peer parity influence participation of women on stack overflow? In: 2017 IEEE Symposium on visual languages and human-centric computing (VL/HCC). IEEE, pp 239–243

  • Fu B, Lin J, Li L, Faloutsos C, Hong J, Sadeh N (2013) Why people hate your app: Making sense of user feedback in a mobile app store. In: 19th International Conference on Knowledge Discovery and Data Mining. ACM, pp 1276–1284

  • Galvis Carreño LV, Winbladh K (2013) Analysis of user comments: an approach for software requirements evolution. In: 35th International Conference on Software Engineering. IEEE, pp 582–591

  • Genderize (2019) Genderize. [Online]. Available: http://www.genderize.io/

  • Google (2020) Google play store categories. [Online]. Available: https://support.google.com/googleplay/android-developer/answer/113475

  • Guzman E, Maalej W (2014) How do users like this feature? a fine grained sentiment analysis of app reviews. In: 22nd International Conference on Requirements Engineering. IEEE, pp 153–162

  • Hannák A, Wagner C, Garcia D, Mislove A, Strohmaier M, Wilson C (2017) Bias in online freelance marketplaces: Evidence from taskrabbit and fiverr. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp 1914–1933

  • Harman M, Jia Y, Zhang Y (2012) App store mining and analysis: Msr for app stores. In: 9th International Conference on Mining Software Repositories, MSR ’12. IEEE, Piscataway, pp 108–111

  • Harrell F E (2001) Regression modeling strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer

  • Hemphill L, Otterbacher J (2012) Learning the lingo? gender, prestige and linguistic adaptation in review communities. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, pp 305–314

  • Hmisc (2020) Harrell miscellaneous. [Online]. Available: http://cran.r-project.org/web/packages/Hmisc/index.html

  • Hong C, Chen ZF, Li C (2017) ``liking” and being ``liked”: How are personality traits and demographics associated with giving and receiving ``likes” on facebook? Comput Human Behav 68:292–299

  • Iacob C, Harrison R (2013) Retrieving and analyzing mobile apps feature requests from online reviews. In: 10th Working Conference on Mining Software Repositories, MSR ’13. IEEE, pp 41–44

  • Ibrahim H, Abdel-Razig S, Stadler DJ, Cofrancesco J, Archuleta S (2019) Assessment of gender equity among invited speakers and award recipients at us annual medical education conferences. JAMA Netw Open 2(11):e1916222–e1916222

  • Islam MR, Zibran MF (2017) Leveraging automated sentiment analysis in software engineering. In: 14th International Conference on Mining Software Repositories. IEEE Press, pp 203–214

  • Jazzy (2017) Jazzy spell checker. [Online]. Available: http://jazzy.sourceforge.net/

  • Karimi F, Wagner C, Lemmerich F, Jadidi M, Strohmaier M (2016) Inferring gender from names on the web: A comparative evaluation of gender detection methods. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp 53–54

  • Khalid H, Nagappan M, Shihab E, Hassan AE (2014a) Prioritizing the devices to test your app on: A case study of android game apps. In: 22nd International Symposium on the Foundations of Software Engineering, pp 370–379

  • Khalid H, Shihab E, Nagappan M, Hassan A E (2014b) What do mobile app users complain about? IEEE Softw 32(3):70–77

    Article  Google Scholar 

  • Kim HW, Lee H, Son J (2011a) An exploratory study on the determinants of smartphone app purchase. In: 11th International DSI and the 16th APDSI Joint Meeting

  • Kim HW, Lee H, Son J (2011b) An exploratory study on the determinants of smartphone app purchase. In: Proceedings of the 11th International Decision Science Institute and the 16th Asia Pacific Decision Sciences Institute Joint Meeting

  • Kim SM, Pantel P, Chklovski T, Pennacchiotti M (2006) Automatically assessing review helpfulness. In: 2006 Conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 423–430

  • Kübler R, Pauwels K, Yildirim G, Fandrich T (2018) App popularity: Where in the world are consumers most sensitive to price and user ratings? J Mark 82(5):20–44

    Article  Google Scholar 

  • Lakoff G (1975) Hedges: A study in meaning criteria and the logic of fuzzy concepts. In: Contemporary research in philosophical logic and linguistic semantics. Springer, pp 221–271

  • Lakoff R (1973) Language and woman’s place. Lang Soc 2(1):45–79

    Article  MathSciNet  Google Scholar 

  • Linares-Vásquez M, Bavota G, Bernal-Cárdenas C, Di Penta M, Oliveto R, Poshyvanyk D (2013) Api change and fault proneness: A threat to the success of android apps. In: 9th Joint Meeting on Foundations of Software Engineering. ACM, pp 477–487

  • Lovins J B (1968) Development of a stemming algorithm. MIT Information Processing Group, Electronic Systems Laboratory

  • Manning CD, Surdeanu M, Bauer J, Finkel J, Bethard SJ, McClosky D (2014) The stanford corenlp natural language processing toolkit. In: 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp 55–60

  • Martin W, Harman M, Jia Y, Sarro F, Zhang Y (2015) The app sampling problem for app store mining. In: 12th Working Conference on Mining Software Repositories. IEEE, pp 123–133

  • Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2016) A survey of app store analysis for software engineering. IEEE Trans Softw Eng PP(99)

  • May A, Wachs J, Hannák A (2019) Gender differences in participation and reward on stack overflow. Empir Softw Eng 24(4):1997–2019

    Article  Google Scholar 

  • McClave J T, Sincich T (2006) Statistics: Technology manual [and CD-ROM]. Pearson, Prentice Hall

    Google Scholar 

  • McMillan C, Poshyvanyk D, Grechanik M, Xie Q, Fu C (2013) Portfolio: Searching for relevant functions and their usages in millions of lines of code. ACM Trans Softw Eng Methodol (TOSEM) 22(4):1–30

    Article  Google Scholar 

  • Merchant A, Shah D, Bhatia GS, Ghosh A, Kumaraguru P (2019) Signals matter: understanding popularity and impact of users on stack overflow. In: The World Wide Web Conference, pp 3086–3092

  • Mukherjee S, Bala P K (2017) Gender classification of microblog text based on authorial style. IseB 15(1):117–138

    Article  Google Scholar 

  • Nguyen TH, Adams B, Hassan AE (2010) Studying the impact of dependency network measures on software quality. In: Proceedings of the 26th International Conference on Software Maintenance. IEEE, pp 1–10

  • Niu H, Keivanloo I, Zou Y (2017) Learning to rank code examples for code search engines. Empir Softw Eng 22(1):259–291

    Article  Google Scholar 

  • Noei E (2018) Succeeding in mobile application markets (from development point of view). PhD thesis, Queen’s University, Canada

  • Noei E, Lyons K (2019) A survey of utilizing user-reviews posted on Google Play Store. In: Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering, pp 54–63

  • Noei E, Da Costa D A, Zou Y (2018) Winning the app production rally. In: 26Th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC/FSE. ACM, New York, pp 283–294

  • Noei E, Zhang F, Wang S, Zou Y (2019a) Towards prioritizing user-related issue reports of mobile applications. Empir Softw Eng 24(4):1964–1996

    Article  Google Scholar 

  • Noei E, Zhang F, Zou Y (2019b) Too many user-reviews, what should app developers look at first? IEEE Transactions on Software Engineering

  • Nord C (2005) Text analysis in translation: Theory, methodology, and didactic application of a model for translation-oriented text analysis, pp 94. Rodopi

  • Otterbacher J (2010) Inferring gender of movie reviewers: exploiting writing style, content and metadata. In: Proceedings of the 19th ACM international conference on Information and knowledge management, pp 369–378

  • Palomba F, Linares-Vásquez M, Bavota G, Oliveto R, Di Penta M, Poshyvanyk D, De Lucia A (2015) User reviews matter! tracking crowdsourced reviews to support evolution of successful apps. In: 31st International Conference on Software Maintenance and Evolution. IEEE, pp 291–300

  • Panichella A, Dit B, Oliveto R, Di Penta M, Poshynanyk D, De Lucia A (2013) How to effectively use topic models for software engineering tasks? an approach based on genetic algorithms. In: 2013 35th International Conference on Software Engineering (ICSE). IEEE, pp 522–531

  • Panichella S, Di Sorbo A, Guzman E, Visaggio CA, Canfora G, Gall HC (2016) Ardoc: app reviews development oriented classifier. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, pp 1023–1027

  • Pinheiro J, Bates D, DebRoy S, Sarkar D, et al. (2007) Linear and nonlinear mixed effects models. R package version 3:57

  • Raftery A E, Lewis S, et al. (1992) How many iterations in the gibbs sampler. Bayesian Stat 4(2):763–773

    Google Scholar 

  • Rajaraman A, Ullman JD (2012) Mining of massive datasets, vol 77. Cambridge University Press, Cambridge

  • Ruiz I J M, Nagappan M, Adams B, Berger T, Dienst S, Hassan A E (2016) Examining the rating system used in mobile-app stores. IEEE Softw 33(6):86–92

    Article  Google Scholar 

  • Scheuerman M K, Spiel K, Haimson O L, Hamidi F, Branham SM (2019) Hci guidelines for gender equity and inclusivity

  • Shull F, Singer J, Sjøberg DI (2007) Guide to Advanced Empirical Software Engineering. Springer, New York

  • Smith BN, Singh M, Torvik VI (2013) A search engine approach to estimating temporal changes in gender orientation of first names. In: Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, pp 199–208

  • Statista (2020) Number of apps available in leading app stores as of march 2017. [Online]. Available: https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/

  • Stephens M (2013) Gender and the geoweb: divisions in the production of user-generated cartographic information. GeoJournal 78(6):981–996

    Article  Google Scholar 

  • Terrell J, Kofink A, Middleton J, Rainear C, Murphy-Hill E, Parnin C, Stallings J (2017) Gender differences and bias in open source: Pull request acceptance of women versus men. PeerJ Comput Sci 3:e111

    Article  Google Scholar 

  • Tian Y, Nagappan M, Lo D, Hassan AE (2015) What are the characteristics of high-rated apps? a case study on free android applications. In: Proceedings of the 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 301–310

  • Topaz C M, Sen S (2016) Gender representation on journal editorial boards in the mathematical sciences. PLoS One 11(8):e0161357

  • Ugoni A, Walker B F (1995) The chi square test: an introduction. COMSIG Rev 4(3):61

    Google Scholar 

  • Van Solingen R, Basili V, Caldiera G, Rombach HD (2002) Goal question metric (gqm) approach. Encyclopedia of software engineering

  • Vasa R, Hoon L, Mouzakis K, Noguchi A (2012) A preliminary analysis of mobile app user reviews. In: Proceedings of the 24th Australian Computer-Human Interaction Conference, pp 241–244

  • Vasilescu B, Capiluppi A, Serebrenik A (2014) Gender, representation and online participation: a quantitative study. Interact Comput 26(5):488–511

    Article  Google Scholar 

  • Villarroel L, Bavota G, Russo B, Oliveto R, Di Penta M (2016) Release planning of mobile apps based on user reviews. In: 38th International Conference on Software Engineering. ACM, pp 14–24

  • Wachs J, Hannak A, Vörös A, Daróczy B (2017) Why do men get more attention? exploring factors behind success in an online design community. In: Eleventh International AAAI Conference on Web and Social Media

  • Wagner C, Graells-Garrido E, Garcia D, Menczer F (2016) Women through the glass ceiling: gender asymmetries in wikipedia. EPJ Data Sci 5(1):5

    Article  Google Scholar 

  • Wais K (2016) Gender Prediction Methods Based on First Names with genderizeR. R J 8(1):17–37

    Article  Google Scholar 

  • Weisberg S (2005) Applied linear regression, vol 528. Wiley

Download references

Author information

Authors and Affiliations

  1. University of Toronto, Toronto, Canada

    Ehsan Noei & Kelly Lyons

Authors
  1. Ehsan Noei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kelly Lyons
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ehsan Noei.

Additional information

Communicated by: Emerson Murphy-Hill

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noei, E., Lyons, K. A study of gender in user reviews on the Google Play Store. Empir Software Eng 27, 34 (2022). https://doi.org/10.1007/s10664-021-10080-8

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-021-10080-8

Keywords

Search

Navigation